Using Feature Selection Approaches to Find the Dependent Features

Dependencies among the features can decrease the performance and efficiency in many algorithms e.g. in classifiers. Therefore, finding the dependent features has become significantly important in many areas. Traditional methods can only find the linear dependencies or the dependencies among few features. In our research, we try to use feature selection approaches for finding dependencies. We generate different datasets with different degrees of complexity using Bayesian Networks. We use and compare Relief, CFS, NBGA and NB-BOA as feature selection approaches to find the dependent features among our artificial data. Unexpectedly, Relief has the best performance in our experiments, even better than NB-BOA, which is a population-based evolutionary algorithm that used the population distribution information to find the dependent features. It may be because some “link strengths” between features are not strong enough for these features to be considered as relevant by NB-BOA. It can be due also to the fact that Naïve Bayes classifier which is used in these wrapper approaches cannot represent the dependencies between features. However, the exact reason for these results still is an open problem for our future work.